FluoroProbe_inspection_2014_2024

Load packages

Build dataframe

devtools::source_url("https://raw.githubusercontent.com/melofton/Reservoirs/refs/heads/master/Scripts/L1_functions/fluoroprobe_create.R")

## ℹ SHA-1 hash of file is "b0108172dbf37004c26d650df6190b9996f7c82e"

## identify latest date for data on EDI (need to add one (+1) to both dates because we want to exclude all possible start_day data and include all possible data for end_day)
package_ID <- 'edi.272.8'
eml <- read_metadata(package_ID)
date_attribute <- xml_find_all(eml, xpath = ".//temporalCoverage/rangeOfDates/beginDate/calendarDate")
first_edi_date <- as.Date(xml_text(date_attribute)) - lubridate::days(1)

## Run Function 
repo_link <- "https://api.github.com/repos/melofton/Reservoirs/git/trees/master?recursive=1"
repo_filepath <- "https://raw.githubusercontent.com/melofton/Reservoirs/refs/heads/master/"
example_file_for_colnames <- "https://raw.githubusercontent.com/CareyLabVT/Reservoirs/refs/heads/master/Data/DataAlreadyUploadedToEDI/CollatedDataForEDI/FluoroProbeData/20140404_CCR_50.txt"
current_year_data_folder <- "Data/DataNotYetUploadedToEDI/FluoroProbe"
historic_data_folder <- "Data/DataAlreadyUploadedToEDI/CollatedDataForEDI/FluoroProbeData"
historic_data_2017 <- "https://raw.githubusercontent.com/CareyLabVT/Reservoirs/refs/heads/master/Data/DataAlreadyUploadedToEDI/CollatedDataForEDI/FluoroProbeData/FP_2017_data/FP_recal_2017.txt"
maintenance_file <- 'https://raw.githubusercontent.com/CareyLabVT/Reservoirs/refs/heads/master/Data/DataNotYetUploadedToEDI/Raw_fluoroprobe/Maintenance_Log_FluoroProbe.csv'
out_file <- NULL
start_date <- first_edi_date
end_date <- Sys.Date() + lubridate::days(1)

# Run function
fp_all <- fluoroprobe_qaqc(repo_link = repo_link,
                 repo_filepath = repo_filepath,
                 example_file_for_colnames = example_file_for_colnames,
                 current_year_data_folder = current_year_data_folder,
                 historic_data_folder = historic_data_folder,
                 historic_data_2017 = historic_data_2017,
                 maintenance_file = maintenance_file,
                 out_file = out_file,
                 start_date = start_date,
                 end_date = end_date) 

## New names:
## Rows: 0 Columns: 32
## ── Column specification
## ──────────────────────────────────────────────────────── Delimiter: "\t" chr
## (32): Date/Time, Green Algae...2, Bluegreen...3, Diatoms...4, Cryptophyt...
## ℹ Use `spec()` to retrieve the full column specification for this data. ℹ
## Specify the column types or set `show_col_types = FALSE` to quiet this message.
## • `Green Algae` -> `Green Algae...2`
## • `Bluegreen` -> `Bluegreen...3`
## • `Diatoms` -> `Diatoms...4`
## • `Cryptophyta` -> `Cryptophyta...5`
## • `#4` -> `#4...6`
## • `#5` -> `#5...7`
## • `#6` -> `#6...8`
## • `Yellow substances` -> `Yellow substances...9`
## • `Green Algae` -> `Green Algae...14`
## • `Bluegreen` -> `Bluegreen...15`
## • `Diatoms` -> `Diatoms...16`
## • `Cryptophyta` -> `Cryptophyta...17`
## • `#4` -> `#4...18`
## • `#5` -> `#5...19`
## • `#6` -> `#6...20`
## • `Yellow substances` -> `Yellow substances...21`

## Warning: `data_frame()` was deprecated in tibble 1.1.0.
## ℹ Please use `tibble()` instead.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

## New names:
## New names:
## • `Green Algae...2` -> `Green Algae...5`
## • `Bluegreen...3` -> `Bluegreen...6`
## • `Diatoms...4` -> `Diatoms...7`
## • `Cryptophyta...5` -> `Cryptophyta...8`
## • `#4...6` -> `#4...9`
## • `#5...7` -> `#5...10`
## • `#6...8` -> `#6...11`
## • `Yellow substances...9` -> `Yellow substances...12`
## • `Green Algae...14` -> `Green Algae...17`
## • `Bluegreen...15` -> `Bluegreen...18`
## • `Diatoms...16` -> `Diatoms...19`
## • `Cryptophyta...17` -> `Cryptophyta...20`
## • `#4...18` -> `#4...21`
## • `#5...19` -> `#5...22`
## • `#6...20` -> `#6...23`
## • `Yellow substances...21` -> `Yellow substances...24`

## Warning: There were 16 warnings in `mutate()`.
## The first warning was:
## ℹ In argument: `GreenAlgae_ugL = as.numeric(`Green Algae...5`)`.
## Caused by warning:
## ! NAs introduced by coercion
## ℹ Run `dplyr::last_dplyr_warnings()` to see the 15 remaining warnings.

## Warning: There was 1 warning in `mutate()`.
## ℹ In argument: `DateTime = as.POSIXct(as_datetime(DateTime, tz = "", format =
##   "%m/%d/%Y %I:%M:%S %p"))`.
## Caused by warning:
## !  11 failed to parse.

## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument

## Rows: 20537 Columns: 29
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: "\t"
## chr   (2): filename, Reservoir
## dbl  (26): green_ugL, cyano_ugL, diatom_ugL, crypto_ugL, yellow_sub_ugL, tot...
## dttm  (1): datetime
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument
## Warning in seq_len(idx): first element used of 'length.out' argument

## Warning: Using an external vector in selections was deprecated in tidyselect 1.1.0.
## ℹ Please use `all_of()` or `any_of()` instead.
##   # Was:
##   data %>% select(colname_start)
## 
##   # Now:
##   data %>% select(all_of(colname_start))
## 
## See <https://tidyselect.r-lib.org/reference/faq-external-vector.html>.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

## Warning: Using an external vector in selections was deprecated in tidyselect 1.1.0.
## ℹ Please use `all_of()` or `any_of()` instead.
##   # Was:
##   data %>% select(colname_end)
## 
##   # Now:
##   data %>% select(all_of(colname_end))
## 
## See <https://tidyselect.r-lib.org/reference/faq-external-vector.html>.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

current_df <- fp_all %>%
  filter(year(DateTime) == 2024)

#write.csv(fp_all, "./FluoroProbe_2014_2024.csv", row.names = FALSE)

Check flags

This section checks to make sure each observation has a data flag. It also checks to make sure the frequency of flags match what we expect to see.

#make sure no NAS in the Flag columns
Flags=current_df%>%
  select(DateTime, starts_with("Flag"))

RowsNA=Flags[!complete.cases(Flags), ] # Keep only the complete rows

#check the flag column
Flags=current_df%>%
  select(starts_with("Flag"))

# Make a table with the number of times a flag was used
for(f in 1:(ncol(Flags))){
  #print(colnames(Flags[f]))
  print(table(Flags[,f], useNA = "always"))
}

## Flag_GreenAlgae_ugL
##    0    3 <NA> 
##  309 7963    0 
## Flag_Bluegreens_ugL
##    0    3 <NA> 
##  309 7963    0 
## Flag_BrownAlgae_ugL
##    0    3 <NA> 
##  309 7963    0 
## Flag_MixedAlgae_ugL
##    0    3 <NA> 
##  309 7963    0 
## Flag_YellowSubstances_ugL
##    0    3 <NA> 
##  309 7963    0 
## Flag_TotalConc_ugL
##    0    3 <NA> 
##  309 7963    0 
## Flag_Temp_C
##    0 <NA> 
## 8272    0 
## Flag_Transmission_perc
##    0    3 <NA> 
##  309 7963    0 
## Flag_RFU_525nm
##    0    3 <NA> 
##  309 7963    0 
## Flag_RFU_570nm
##    0    3 <NA> 
##  309 7963    0 
## Flag_RFU_610nm
##    0    3 <NA> 
##  309 7963    0 
## Flag_RFU_370nm
##    0    3 <NA> 
##  309 7963    0 
## Flag_RFU_590nm
##    0    3 <NA> 
##  309 7963    0 
## Flag_RFU_470nm
##    0    3 <NA> 
##  309 7963    0

Plot all casts individually - reviewers should not run this!

Define heatmap function

flora_heatmap <- function(fp_data, reservoir, year, site, z){
  
  #subset to relevant data
  fp <- fp_data %>%
    filter(Reservoir == reservoir & year(DateTime) == year & Site == site) %>%
    select(CastID, DateTime, Depth_m, {{z}}) 
  
  #slice by depth for each reservoir
  if (reservoir == "FCR"){
    
    if(site == 50){
       depths = seq(0.1, 9.3, by = 0.3)
    } else if(site == 40){
      depths = seq(0.1, 8.5, by = 0.3)
    } else if(site == 30){
      depths = seq(0.1, 7, by = 0.3)
    } else if(site == 20){
      depths = seq(0.1, 4.5, by = 0.3)
    } else if(site == 10){
      depths = seq(0.1, 3.5, by = 0.3)
    }
  
  df.final<-data.frame()
  
  for (i in 1:length(depths)){
    
fp_layer <- fp %>% 
  group_by(CastID) %>% 
  slice(which.min(abs(as.numeric(Depth_m) - depths[i])))

# Bind each of the data layers together.
df.final = bind_rows(df.final, fp_layer)

}


} else if (reservoir == "BVR"){
  
  depths = seq(0.1, 10, by = 0.3)
  df.final<-data.frame()
  
  for (i in 1:length(depths)){
    
    fp_layer<-fp %>% group_by(CastID) %>% slice(which.min(abs(as.numeric(Depth_m) - depths[i])))
    
    # Bind each of the data layers together.
    df.final = bind_rows(df.final, fp_layer)
    
  }
  
} else if(reservoir == "CCR"){
  
  depths = seq(0.1, 20, by = 0.3)
  df.final<-data.frame()
  
  for (i in 1:length(depths)){
    
    fp_layer<-fp %>% group_by(CastID) %>% slice(which.min(abs(as.numeric(Depth_m) - depths[i])))
    
    # Bind each of the data layers together.
    df.final = bind_rows(df.final, fp_layer)
    
  }
  } else if(reservoir == "GWR"){
  
  depths = seq(0.1, 12, by = 0.3)
  df.final<-data.frame()
  
  for (i in 1:length(depths)){
    
    fp_layer<-fp %>% group_by(CastID) %>% slice(which.min(abs(as.numeric(Depth_m) - depths[i])))
    
    # Bind each of the data layers together.
    df.final = bind_rows(df.final, fp_layer)
    
  }
  } else if(reservoir == "SHR"){
  
  depths = seq(0.1, 30, by = 0.3)
  df.final<-data.frame()
  
  for (i in 1:length(depths)){
    
    fp_layer<-fp %>% group_by(CastID) %>% slice(which.min(abs(as.numeric(Depth_m) - depths[i])))
    
    # Bind each of the data layers together.
    df.final = bind_rows(df.final, fp_layer)
    
  } 
  
  }
  
  #wrangle final dataframe for plotting
  # Re-arrange the data frame by date
  fp_new <- arrange(df.final, DateTime)

  # Round each extracted depth to the nearest 10th. 
  fp_new$Depth_m <- round(as.numeric(fp_new$Depth_m), digits = 0.5)
  
  # Convert to DOY
  fp_new$DOY <- yday(fp_new$DateTime)
  
  fig_title <- paste(reservoir, year, "Site", site, z, sep = " ")
  
  interp <- interp(x=fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]),
                      xo = seq(min(fp_new$DOY), max(fp_new$DOY), by = .1), 
                      yo = seq(min(fp_new$Depth_m), max(fp_new$Depth_m), by = 0.01),
                      extrap = T, linear = T, duplicate = "strip")
interp <- interp2xyz(interp, data.frame=T)
  
  p1 <- ggplot(interp, aes(x=x, y=y))+
  geom_raster(aes(fill=z))+
  scale_y_reverse(expand = c(0,0))+
  scale_x_continuous(expand = c(0, 0)) +
  scale_fill_gradientn(colours = blue2green2red(60), na.value="gray")+
  labs(x = "Day of year", y = "Depth (m)", title = fig_title,fill=expression(paste(mu,g/L)))+
  theme_bw()

print(p1)

}

Visualize current year at a glance

flora_heatmap(fp_data = current_df, reservoir = "FCR", year = 2024, site = 50, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

flora_heatmap(fp_data = current_df, reservoir = "FCR", year = 2024, site = 40, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!
## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

flora_heatmap(fp_data = current_df, reservoir = "FCR", year = 2024, site = 30, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!
## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

flora_heatmap(fp_data = current_df, reservoir = "FCR", year = 2024, site = 20, z = "TotalConc_ugL")

flora_heatmap(fp_data = current_df, reservoir = "FCR", year = 2024, site = 10, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!
## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

flora_heatmap(fp_data = current_df, reservoir = "BVR", year = 2024, site = 50, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!
## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

Visualize a previous year at a glance

flora_heatmap(fp_data = fp_all, reservoir = "FCR", year = 2016, site = 50, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

flora_heatmap(fp_data = fp_all, reservoir = "BVR", year = 2016, site = 50, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!
## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

flora_heatmap(fp_data = fp_all, reservoir = "CCR", year = 2016, site = 50, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!
## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

flora_heatmap(fp_data = fp_all, reservoir = "SHR", year = 2016, site = 50, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!
## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

flora_heatmap(fp_data = fp_all, reservoir = "GWR", year = 2016, site = 50, z = "TotalConc_ugL")

## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## collinear points, trying to add some jitter to avoid colinearities!
## Warning in interp(x = fp_new$DOY, y = fp_new$Depth_m, z = unlist(fp_new[z]), :
## success: collinearities reduced through jitter

Check to make sure that what is in the maintenance log was actually removed

Look at the last rows of the maintenance log

We want to make sure that our maintenance log actually worked and took out the values or changes those it was supposed to

## Rows: 17 Columns: 11
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (7): Reservoir, DataStream, TIMESTAMP_start, TIMESTAMP_end, start_parame...
## dbl (2): Site, flag
## lgl (2): Depth, update_value
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

## # A tibble: 6 × 11
##   Reservoir  Site Depth DataStream TIMESTAMP_start TIMESTAMP_end start_parameter
##   <chr>     <dbl> <lgl> <chr>      <chr>           <chr>         <chr>          
## 1 BVR          50 NA    Flora      2019-09-11 00:… 2019-09-11 2… Temp_C         
## 2 FCR          50 NA    Flora      2019-09-02 00:… 2019-09-02 2… Temp_C         
## 3 FCR          50 NA    Flora      2019-08-28 00:… 2019-12-31 2… Transmission_p…
## 4 BVR          50 NA    Flora      2019-08-28 00:… 2019-12-31 2… Transmission_p…
## 5 CCR          50 NA    Flora      2019-08-28 00:… 2019-12-31 2… Transmission_p…
## 6 FCR          50 NA    Flora      2020-08-24 00:… 2020-08-24 2… Temp_C         
## # ℹ 4 more variables: end_parameter <chr>, flag <dbl>, update_value <lgl>,
## #   notes <chr>

Reservoir	Site	Depth	DataStream	TIMESTAMP_start	TIMESTAMP_end	start_parameter	end_parameter	flag	update_value	notes
BVR	50	NA	Flora	2019-09-11 00:00:00 EDT	2019-09-11 23:59:59 EDT	Temp_C	Temp_C	2	NA	bad temperature data
FCR	50	NA	Flora	2019-09-02 00:00:00 EDT	2019-09-02 23:59:59 EDT	Temp_C	Temp_C	2	NA	bad temperature data
FCR	50	NA	Flora	2019-08-28 00:00:00 EDT	2019-12-31 23:59:59 EDT	Transmission_perc	Transmission_perc	2	NA	bad transmission data (all 100%)
BVR	50	NA	Flora	2019-08-28 00:00:00 EDT	2019-12-31 23:59:59 EDT	Transmission_perc	Transmission_perc	2	NA	bad transmission data (all 100%)
CCR	50	NA	Flora	2019-08-28 00:00:00 EDT	2019-12-31 23:59:59 EDT	Transmission_perc	Transmission_perc	2	NA	bad transmission data (all 100%)
FCR	50	NA	Flora	2020-08-24 00:00:00 EDT	2020-08-24 23:59:59 EDT	Temp_C	Temp_C	2	NA	bad temperature data

Check the that the columns have flags

Look at the first few rows of the data frame and check that the observations after the TIMESTAMP_start are flagged

Look at the first 5 rows for that time

Reservoir	DateTime	Temp_C	Flag_Temp_C
FCR	2020-08-24 15:53:03	NA	2
FCR	2020-08-24 15:53:05	NA	2
FCR	2020-08-24 15:53:07	NA	2
FCR	2020-08-24 15:53:10	NA	2
FCR	2020-08-24 15:53:12	NA	2
FCR	2020-08-24 15:53:15	NA	2

Look at the last 6 rows for the maintenance time

Make sure the observations are flagged

Reservoir	DateTime	Temp_C	Flag_Temp_C
FCR	2020-08-24 15:55:06	NA	2
FCR	2020-08-24 15:55:09	NA	2
FCR	2020-08-24 15:55:11	NA	2
FCR	2020-08-24 15:55:14	NA	2
FCR	2020-08-24 15:55:16	NA	2
FCR	2020-08-24 15:55:18	NA	2

Make site description file

 # These lines of code make the csv of the site descriptions with lat and long
 # MEL You don't need to run this if you already have the file I believe?

  # # Use Gsheet because you don't need to authenticate it. 
  # sites <- gsheet::gsheet2tbl("https://docs.google.com/spreadsheets/d/1TlQRdjmi_lzwFfQ6Ovv1CAozmCEkHumDmbg_L4A2e-8/edit#gid=1244423834")
  # #data<- read_csv("YOUR DATA.csv")# Use this if you read in a csv
  # data <- current_df #This is the line you need to modify!
  # trim_sites = function(data,sites){
  #   data_res_site=data%>% #Create a Reservoir/Site combo column
  #     mutate(res_site = trimws(paste0(Reservoir,Site)))
  #   sites_merged = sites%>% #Filter to Sites that are in the dataframe
  #     mutate(res_site = trimws(paste0(Reservoir,Site)))%>%
  #     filter(res_site%in%data_res_site$res_site)%>%
  #     select(-res_site)
  # }
  # sites_trimmed = trim_sites(data,sites) 
  # write.csv(sites_trimmed,"site_descriptions.csv", row.names=F)# Write to file

Download the QAQC function

We will put it in the folder where the EDI production files are to make sure that it happens and to get the most recent version

download.file("https://raw.githubusercontent.com/melofton/Reservoirs/master/Scripts/L1_functions/fluoroprobe_create.R", "FluoroProbe_qaqc_2014_2024.R")